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REVIEW OF BETTER EVIDENCE, BETTER 
CHOICES, BETTER SCHOOLS 


Terri S. Wilson, University of Colorado Boulder 


I. Introduction 


The 2015 “Every Student Succeeds Act” (ESSA) has been celebrated for returning critical 
decision-making authority to states and districts.: Unlike its earlier predecessor, the 2001 
“No Child Left Behind” Act, ESSA grants states more flexibility and control over the design 
of accountability and assessment systems, teacher certification and evaluation, among other 
issues. While the consequences of this flexibility remain uncertain, states are now rede- 
signing a range of policies in advance of ESSA’s full implementation in 2017-2018. So far, 
attention has largely been focused on accountability policies, but another crucial change 
involves ESSA’s shift in the standards of evidence required to justify these state and local 
policy decisions. 


Where NCLB famously required reform strategies be based on “scientifically-based re- 
search,” ESSA prioritizes a somewhat more flexible “evidence-based” standard. Under 
NCLB, the goal of “scientifically-based research” was defined in terms of a single method- 
ological approach: randomized controlled experiments, the so-called “gold standard” of ev- 
idence.? While some advocates saw this change as an overdue corrective to an undisciplined 
field, the singular focus on randomized controlled trials (RCTs) was widely criticized by 
the research community as an overly-prescriptive and rigid definition of research quality.’ 
States will now be able to choose context-relevant interventions, but ESSA requires that 
these interventions be “evidence-based,” and specifies the kinds of evidence that states can 
use in justifying their approaches to school improvement.‘ Although still prescriptive about 
acceptable kinds of evidence, this shift asks states to take on new responsibility for evaluat- 
ing the relative quality and strength of evidence in support of various reform strategies. This 
is a different kind of challenge. As Feuer, Towne and Shavelson note, “Almost everyone can 
appreciate, intuitively, the advantages of evidence-based policy; it is another matter entirely 
to make this concept clear, operational, and valid.”5 


This report steps into these debates. “Better Evidence, Better Choices, Better Schools: State 
Supports for Evidence-Based School Improvement and the Every Student Succeeds Act” is 
a new report jointly released by the Center for American Progress (CAP) and the Knowledge 
Alliance.®° The second in a series of planned reports about the implementation of ESSA, this 
25-page paper has two central aims: (1) to clarify the new definition of “evidence-based” em 
ployed in ESSA, and (2) to offer guidance to state education agencies in implementing the 
“evidence-based school improvement practices” called for in ESSA. The authors are Steve 
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Fleischman, CEO of Education Northwest, and current chairperson of Knowledge Alliance, 
with co-authors Caitlin Scott, a “practice expert on education policy, research, and evaluation” 
at Education Northwest, and Scott Sargrad, the Managing Director of K-12 Education Policy at 
the Center for American Progress. 


Although this report underscores the significant challenges facing state education agencies in 
this new policy climate, it provides relatively thin and unsubstantiated advice to guide pol- 
icymakers or practitioners. In this review, I summarize the findings and conclusions of the 
report, review its rationale, use of research, and methodological approach, and evaluate its 
usefulness for guiding policy and practice. 


Il. Findings and Conclusions of the Report 


The report begins by noting that “ESSA offers state education agencies significant opportuni- 
ties to use evidence to support the improvement of schools and ensure better outcomes for all 
students.”” Seeking to help states leverage these new opportunities, the report has two major 
goals: to clarify the new definition of “evidence-based” in ESSA, and offer guidance to state 
education agencies in implementing ESSA’s “evidence-based school improvement practices.” 
As the authors note in the introduction, these decisions come into play with the new flexibil- 
ity granted to states for funding dedicated to school improvement. Under ESSA, states may 
now use up to 7 percent of their Title I funds to support school improvement, and can set an 
additional 3 percent of Title I funds aside for “direct student services,” including tutoring and 
credit recovery. Where NCLB specified particular school improvement strategies, states now 
have the flexibility to choose among many reform approaches and interventions, provided that 
these are “evidence-based.” These choices offer new challenges (and perhaps new opportuni- 
ties) for states. 


A brief introductory “Author’s Note,” makes three moves to limit the scope of the inquiry. The 
authors first note that they are primarily focused on the use of the term “evidence-based” in 
relation to “state and district school improvement activities,” and that more detailed defi- 
nitions of the term can be found in guidance from the U.S. Department of Education. The 
authors also caution that their report, “does not attempt to provide a detailed list of concep- 
tual and methodological issues related to the implementation of ESSA evidence tiers.” Here, 
they briefly reference—but do not summarize—a recent RAND report that has raised “some 
of these early questions and issues.”® Third, while noting that ESSA implementation is in the 
early stages, their report focuses on “identifying important considerations for state education 
agencies as they work collaboratively with local districts to implement evidence-based policy 
and practice.” 


My review will focus largely on how well the report does what it aims to do: namely, identify 
some of the considerations facing state education agencies (SEAs) as they move forward with 
ESSA implementation. Yet, one critical question is how easily such policy concerns can be 
separated from methodological issues. Part of the difficult work facing SEAs and local districts 


http://nepce.colorado.edu/thinktank/review-CAP 4 of 19 


involves making complex decisions about which reforms to pursue. These decisions—ac- 
cording to ESSA—must be “evidenced-based,” meaning that methodological issues about the 
sources and availability of evidence and what counts as evidence cannot be as easily bracket- 
ed from issues of decision-making. While “Better Evidence, Better Choices, Better Schools” 
largely skims over methodological questions about evidence, such questions are crucial to 
informing state-level decision-making. 


The report begins with a brief summary of the evidence-based approaches defined in the 
Every Student Succeeds Act. The report quickly recaps the three “levels of evidence to use 
when choosing an approach to school improvement” (p. 4). These evidence levels, the au- 
thors note, are “commonly referred to” as: Tier I, “strong evidence”; Tier II, “moderate ev- 
idence” and Tier III, “promising evidence.” A text box on the next page helpfully reprints a 
key passage (Section 8101), from the federal legislation that offers more detail. Indeed, each 
tier of evidence in ESSA is linked to a specific methodology. The law notes that the term 
“evidence-based” means: 


“an activity, strategy, or intervention that (i) demonstrates a statistically signif- 
icant effect on improving student outcomes or other relevant outcomes based 
on—(I) strong evidence from at least 1 well-designed and well-implemented 
experimental study; (II) moderate evidence from at least 1 well-designed and 
well-implemented quasi-experimental study; or (III) promising evidence from 
at least 1 well-designed and well-implemented correlational study with statis- 
tical controls for selection bias; or (ii) (1) demonstrates a rationale based on 
high-quality research findings or positive evaluation that such activity, strate- 
gy, or intervention is likely to improve student outcomes or other relevant out- 
comes; and (II) includes ongoing efforts to examine the effects of such activity, 
strategy, or intervention.? 


ESSA’s definition, while technical, is actually quite a bit more specific than the report about 
the kinds of methodological approaches that qualify as “evidence-based” and to what degree. 
The authors note that, “the creation of these tiers implies that—all things being equal—dis- 
tricts and schools should use improvement approaches with the strongest level of evidence.”?° 
Indeed, ESSA actually requires states to do so. Recent non-regulatory guidance from the De- 
partment of Education underscored that state decision-makers should consider the “entire 
body of relevant evidence,” but prioritize “strong evidence,” defined as significant and posi- 
tive results from at least one “well-designed and well-implemented experimental study (e.g., 
a randomized controlled trial)” or “moderate evidence,” defined as significant and positive 
results from at least one well-designed and well-implemented quasi-experimental study." 


Only if there is no available “strong” or “moderate” evidence should states and districts con- 
sider “promising” evidence, defined by ESSA as “at least one well-designed and well-imple- 
mented correlational study with statistical controls for selection bias on the intervention.” 
In addition to these three tiers of evidence, ESSA does include another category, for use 
in limited circumstances. This new category requires that states or districts offer an evi- 
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dence-based “rationale” for an intervention, along with an ongoing commitment to evaluate 
that intervention. This is indeed the most flexible and intriguing of the new categories, but 
the authors note that approaches supported by “fourth tier” evidence cannot be used for the 
purpose of measuring school improvement supported by Title I school improvement funds 
under Section 1003(a). 


After outlining the new model of “evidence-based” in ESSA, the report contrasts this view 
with the definition of “scientifically based research” (SBR) posed in NCLB. The authors note— 
while not citing other researchers—that SBR “limited acceptable evidence to well-conducted 
randomized control trials, or RCTs—also known as experiments—and certain high-quality 
studies with quasi-experimental designs, or QEDs.” The authors refer to some of the dif- 
ficulties posed by this narrower definition of research: in their analysis, these challenges 
include few available studies, limited effect sizes, and that SBR asked policymakers to focus 
“too narrowly” on research design issues. While not cited by the report, a number of scholars 
have pointed to specific challenges associated with the single-minded focus on RCTs in ed- 
ucational research. For instance, in an intriguing recent study, Allan Ginsburg and Marshall 
Smith (both strong advocates of experimental methodology) outlined a variety of non-selec- 
tion bias threats in the RCT studies of math curricula that are featured on the What Works 
Clearinghouse.” These threats include flaws in implementation, uneven implementation 
between control and experimental classrooms, as well as the fact that several studies were 
sponsored by the developers of the curriculum being evaluated. The authors underscore that 
random samples cannot fully account for “the reality that complex social systems such as 
classrooms and schools often create major challenges for applying RCTs.”'3 While research- 
ers have noted some of the challenges involved in RCTs in education, this research is largely 
not noted in the report. 


While the authors pose a strong contrast between NCLB’s view of “scientifically based re- 
search” and the new “evidence-based” standard in ESSA, it is important to note that, in prac- 
tice, there is not much difference between the two approaches. While ESSA does add a new 
tier of “promising” evidence, it still prioritizes and preserves NCLB’s emphasis on a small 
band of accepted research methods: RCTs (the new Tier I) and QEDs (the new Tier IT). It is 
likely that encouragement (and enforcement) of acceptable evidence levels will rest with the 
states; perhaps they will be more open to drawing on Tier III evidence, and, in some cases, 
exploring Tier IV evidence. But at this point, the changing standards of evidence in ESSA 
look quite thin indeed, and are much more modest than the ones celebrated in the report. 
This calls into question some of the sweeping contrasts made in the report. Indeed, in plac- 
es, the report veers into evangelism for advances in evidence-based research. For example, 
the authors write in the introduction: 


The Center for American Progress and Knowledge Alliance believe that the evi- 
dence-based provisions of ESSA are a significant advancement in promoting the 
use of evidence to support schools in need of improvement and to achieve better 
student outcomes. CAP and Knowledge Alliance also believe that the shift away 
from federal mandates and toward greater state and local autonomy is a move in 
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the right direction. Together, these changes have the potential to drive a new era 
of school improvement in the United States. (p. 2, emphasis added) 


In order to support some of the contrasts they develop between NCLB and ESSA, the authors 
reference researchers Kara Finnigan and Alan Daly who, they write, “have pointed out that 
the No Child Left Behind-era emphasis on scientifically based research has not resulted in 
a unified approach to ensuring that all students have effective schools.” The specific source 
by Finnigan and Daly is not included or cited in the report. Finnigan and Daly have indeed 
pointed to ways in which NCLB’s focus on scientifically based research has not led to wide 
school improvement. But their research underscores the complex, socially constructed ways 
that district and school leaders share and use data, and the importance of relationships of 
trust in school reform efforts.4 


The authors, rather than taking up these claims, simply state, “Improving education will 
require that policymakers and educators alike reject whims, fads, and promotional claims in 
favor of reliable evidence.” This is a troubling over-simplification of Finnigan and Daly’s 
position, but also misses the significance of their research—and other scholarly work—on 
how stakeholders use and evaluate research in school reform efforts.*° This emerging re- 
search underscores that it is not so simple as urging policymakers and educators to “re- 
ject whims” in favor of “reliable evidence.” In contrast, Michael McPherson summarizes the 
complexity and depth of this process: 


“This problem of how research becomes effective in practice is itself a social 
science question of considerable depth and complexity that deserves study in 
its own right. The paths by which research knowledge winds its way into the 
daily life of educational organizations, the paths by which practitioner knowl- 
edge is brought to bear and made to count in the research process, and the paths 
by which researchers become interested in problems of genuine importance to 
practice are complex and hard to understand and warrant systematic analysis 
and reflection.”” 


While favoring flexibility for local decision-makers earlier in the report,'® the authors shift 
gears in this middle section, arguing that “struggling schools should not be left to their own 
devices to improve by trial and error—not when there are approaches that have a legitimate 
evidence base and not when time and resources are at a premium.”*? To make this point, 
they quote Rebecca Herman and colleagues, but do not name or cite a particular source. This 
argument is left undeveloped and unsubstantiated, but raises a number of questions about 
the report’s support for expanding the criteria of acceptable evidence and supporting states 
and districts to design context-specific school improvement strategies. 


After these very general arguments, the latter part of the report focuses on a series of recom- 
mendations (geared to SEAs) that depart somewhat sharply from the discussion of evidence 
in the first part of the report. The recommendations are derived from a focus group con- 
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ducted with former state and local education agency employees, informal interviews, and 
“the reflections of the authors based on their school improvement work over the past two 
decades.”*° The recommendations include the following points: 


1. Define roles in the policy implementation process and create a theory of change 


2. Support the use of evidence clearinghouses to identify promising approaches, 
rather than creating new state-approved lists 


3. Encourage robust decision-making processes 

4. Help ensure high-quality services from improvement providers 

5. Promote and provide frequent, accurate, and timely communication 

6. Partner with intermediaries to promote and support effective implementation 
7. Facilitate effective implementation in districts, schools and classrooms 


8. Promote continuous improvement and collective learning. 


In addition, a separate text box in the report summarizes a 2013 paper by Education North- 
west staff that looked closely at the flawed implementation of an intervention in one dis- 
trict.21 This earlier paper’s literature review, focusing on the factors that contribute to im- 
plementation success, is briefly summarized in bullet points in the new report.”? The authors 
also note three issues raised by district stakeholders in this previous study, including the 
importance of following district procedures for adopting new programs, reviewing evidence 
on program effectiveness and ensuring that a new program aligns with district policies and 
goals. 


After short summaries of each recommendation, a brief two-paragraph conclusion restates 
the authors’ faith in evidence-based approaches, especially ones that can be flexible to lo- 
cal needs. The report concludes: “It cannot be known with absolute certainty what results 
will be achieved when school leaders select evidence-based school improvement approaches 
because no one can predict the future with 100 percent accuracy. What is known, however, 
is that evidence-based approaches—when wisely chosen and implemented to meet specific 
local needs—offer the best chance of success.”3 


lll. The Report’s Rationale for its Findings and Conclusions 


The report has two distinct aims and sections, which have slightly different flaws. The first 
section of the report aims to clarify the new definition of “evidence-based” employed in 
ESSA for policymakers and practitioners. The report does, indeed, reproduce the relevant 
sections from the new law, but does little to explicate the relevance of these definitions for 
practice. More so, the brief explanation offered minimizes the significance of the tiers of ev- 
idence: for ESSA, some forms of evidence should be prioritized above others. The authors do 
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note, “that—all things being equal—districts and schools should use improvement approach- 
es with the strongest level of evidence.” This implies that districts and schools have consid- 
erably more autonomy over these decisions than may be the case. While ESSA does include 
a third tier of evidence (reserved for correlational studies with acceptable controls), it also 
underscores that such studies should only be used if other forms of evidence are unavailable. 


In this sense, the report exaggerates the distinctions between NCLB’s standard of scientifi- 
cally-based research and ESSA’s guidance on evidence-based practice. The use of the term 
“evidence-based” in ESSA is not actually a significant departure from recent federal educa- 
tion policy. Indeed, in a post-NCLB era, the Department of Education has prioritized the use 
of the term “evidence-based” in many of its discretionary grant programs and funding com- 
petitions, including “Race to the Top,” which asked for states to employ “evidence-based” 
interventions and approaches to school improvement. In many ways, the “new” definition 
of evidence-based research is a continuation of previous requirements from the Department 
of Education, including its i3 grant programs, as well as other federal programs.** Federal 
efforts to convert evidence-based concepts into policy were detailed in the 2014 book, Show 
Me The Evidence: Obama’s Fight for Rigor and Results in Social Policy.?5 While ESSA does 
rewrite policy in decisive ways, the law’s shifting standards of evidence are both more grad- 
ual and mild than presented in this report. 


The second major part of the report turns to prac- 
ne : , tice. Here, the report makes eight very general rec- 
distinctions between NCLB's ommendations about the collaborative implemen- 
standard of scientifically- tation of intervention programs. While the authors 
based research and ESSA’s _ pote that these findings are based on a focus group 
guidance on evidence-based and informal interviews, little to no direct evidence 
practice. is included to support these recommendations. The 
recommendations often read as relatively non-con- 
troversial aphorisms (e.g., “states should communicate about the importance of using ev- 
idence for improvement,” or “state education agencies should reach out to their education 
service agencies”). While not bad advice, certainly, the recommendations are disconnected 
from any data sources, research evidence, or many of the difficult complexities of practice 
that—while mentioned—remain unexplored. 


The report exaggerates the 


Chief among these unaddressed complexities is how districts and schools will be able to 
wade through various sources of evidence, make judgments about the strength and reliabil- 
ity of that evidence, and use that evidence to justify their choices of interventions. The au- 
thors do briefly note the existence of various resources designed to help practitioners eval- 
uate the effectiveness of educational interventions, such as the What Works Clearinghouse 
(WWC) and the Regional Education Laboratories (RELs). They also argue that states should 
help support local decision-making, not create lists of approved programs. Yet, beyond ex- 
horting practitioners to make use of these existing resources, they offer little advice for how 
local decision-makers might evaluate these available resources in light of these changing 
standards of evidence. More interestingly, the authors contend that schools and districts 
should not only consider the strength of the available evidence, but also think about the fit 
and alignment of an intervention in its local context. Here, their position moves away—at 


http://nepce.colorado.edu/thinktank/review-CAP 9 of 19 


least to some degree—from the specific tiers of evidence posed by ESSA, and encourages 
local stakeholders to make decisions that might incorporate a wider view of evidence. Yet 
this shift raises real challenges for local decision-makers: how should they integrate the 
available evidence base with insights from their local contexts? When and how might it be 
appropriate to move away from the Tier I and Tier II evidence base, and to what degree? 


These are challenging questions, and ones that call for more detailed analysis. At the start 
of the report, the authors briefly reference one of the major challenges facing SEAs and dis- 
tricts. They ask, somewhat rhetorically, “Entering the era of evidence-based school improve- 
ment, state education agencies are asking: Will the evidence-based policy elements of ESSA 
be any different, or will education product and service providers simply replace “scientifi- 
cally based” with “evidence based” in their promotional materials? Will anything change for 
the better?”?° Here, the authors raise—but do not address—a key new challenge for schools 
and districts: considering a huge amount of possible interventions in a highly marketized 
environment, where curriculum developers have strong incentives to establish themselves 
as “evidence-based.” District and school officials are often bombarded with marketing ma- 
terials for various curricula and intervention programs; this material is, of course, often 
simpler and easier to understand than studies about the relative effectiveness of materials 
available from research-based sources, including the WWC. Beyond being able to evaluate 
the strength of evidence, an additional challenge may be helping local decision-makers be- 
come critical and skeptical consumers of research. In addition to effect sizes, they may also 
need to think about the sponsorship of research studies, as well as distinctions between 
scholarly research and advocacy pieces. Without such conversations, the new—and com- 
plex—landscape of choices may merely advantage private curriculum developers and inter- 
vention models. 


IV. The Report’s Use of Research Literature 


While noting the importance of research evidence, the report’s use of research literature is 
weak. As noted earlier, the authors do reproduce key sections from ESSA, but do not explain 
or contextualize these passages with any secondary literature. While seeking to distinguish 
the new “evidence-based” standard from NCLB’s focus on “scientifically based research,” the 
authors do not draw on the sustained and thoughtful considerations of these differences that 
have been posed by researchers. Instead, the authors make claims that rest on conjecture 
and supposition (e.g., “It seemed over the past decade that decision-makers were constantly 
hearing from researchers and experts that almost nothing works,” or “This new strategy for 
using evidence seems more suited to the reality of the existing research base in education 
and the choices available to decision-makers”).?7 


In general, there are very few research-based sources cited to support the claims in this re- 
port. Only 18 end notes are provided, drawn from 11 unique sources. Four of the notes sim- 
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ply refer to laws and two more are explanatory notes to the text. Only a few research-based 
articles are cited. Also troubling were the numerous references to authors who are either not 
cited in the bibliography, or are cited to a secondary source. For example, the authors quote 
Sam Stringfield and Rebecca Herman, but without citing the sources of those quotations. 
Likewise, researchers are invoked with citation (e.g., Finnigan and Daly), or only through a 
secondary source (e.g., Dobbie and Fryer’s work is noted, but the source cited is a CAP re- 
port).?8 Without a stronger basis in the available literature, many of the report’s conclusions 
remain assertions, rather than arguments supported by evidence. 


V. Review of the Report’s Methods 


The authors state that they based their eight recommendations “on a March 2016 focus group 
of former state and local education agency employees; informal interviews with education 
administrators and improvement experts, also conducted in March 2016; and the reflections 
of the authors based on their school improvement work over the past two decades.”?? Beyond 
this sentence, however, there is no explanation of the methods they used. For instance, how 
many participants were part of the focus group? The informal interviews? How were such 
participants identified and recruited? How were these interviews and the focus group con- 
ducted, and by who? What kinds of data were gathered and how was this data analyzed? In 
what ways do the recommendations draw on this data? 


Beyond these questions, there is little to any mention of any of the results from these in- 
terviews or focus groups. There are no direct quotes from participants, references to key 
examples, or any summary of the findings from these efforts. Here, it is worth returning to 
the standards of evidence championed throughout the report. The methods employed by the 
researchers would certainly not qualify in any of the tiers of evidence, and also fall short of 
well-recognized standards for quality research, regardless of design. 


VI. Review of the Validity of the Findings and Conclusions 


As noted earlier, the major findings of the report include eight general recommendations for 
how state and local decision-makers might implement interventions in light of the changes 
proposed in ESSA. The recommendations lack a grounding in both the available research 
base and in the empirical study conducted by the report’s authors. Beyond a lack of evidence, 
the conclusions remain curiously disconnected from the first part of the report, which re- 
views the shifting standards of research evidence. The report’s recommendations and con- 
clusions might have been stronger and more specific if tied to the specific changes posed in 
ESSA. As is, the report is organized into two separate and disconnected pieces, which do not 
logically connect to one another. 
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Vil. Usefulness of the Report for Guidance of Policy and Practice 


It’s important to underscore that this is a preliminary report and the future directions of 
ESSA are still very much up in the air. Indeed, the details of implementation will continue to 
develop—and vary by state—throughout this year. For this reason, some of the uncertainty or 
vagueness in the report’s recommendations quite likely (and legitimately) reflect the uncer- 
tainty of the policy field. Yet, it is worth thinking about the usefulness of the report against 
its own two main aims: (1) to clarify the new definition of “evidence-based” employed in 
ESSA, and (2) to offer guidance to practitioners in making evidence-based decisions. As not- 
ed above, the report does not offer much clarity on the definition of evidence posed by ESSA, 
and—in some cases—overplays its contrast with previous standards of evidence offered in 
NCLB and subsequent federal programs. In addition, the guidance offered to practitioners 
is vague and unsubstantiated. 


The report also brushes over—and at times conflates—two very different views of how ev- 
idence interacts with policy and practice. These different narratives are not made explicit 
in the report, but it is worth noting how the two may in fact come into conflict with one an- 
other. In one view, the goal of research is to develop unbiased and rigorous evaluations of 
interventions, often through experimental or quasi-experimental designs.°° Here, the goal is 
to find “proven strategies” to address educational challenges. Not all strategies will produce 
results; indeed, many interventions evaluated by the WWC have been found to have null ef- 
fects on student outcomes.*! The goal, however, is to find at least a few positive interventions 
and then replicate—or take them “to scale” across different educational contexts. This ratio- 
nale has been broadly influential in shaping the design and policies of federally-sponsored 
educational research, including the investment in the What Works Clearinghouse (WWC). 
While successful interventions have been identified through these efforts, other evaluations 
have pointed to the complexity of evaluating curricular interventions and the need for more 
replication studies across unique contexts.°” 


A contrasting perspective challenges the paradigm of moving from research to practice, 
and calls instead for thinking about the interaction between research and practice, as well 
as for the diverse ways that research is acquired and used.33 As Nancy Cartwright argues, 
“the project needs to be approached from the point of view of the evidence user, not the 
evidence producer.” Scholars have pointed to the deeply contextual nature of policy and 
program implementation, and how local contexts can be powerful sources of research evi- 
dence and educational improvement. Fred Erickson and Kris Gutiérrez, for instance, have 
argued that rigorous evaluation must involve careful, qualitative documentation of what 
actually happens in a given intervention, and, more importantly, how students and teach- 
ers adapt to—and also creatively alter—interventions in dynamic local contexts.*5 Building 
on these insights, a number of scholars and federal programs have started to explore these 
“persistent problems of practice,” focusing on how interventions become relevant to specific 
local contexts.°° 


This alternate perspective has shaped emerging research into the complex and interactive 
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relationship between research and practice. For instance, experiments in design-based im- 
plementation research (DBIR) and Research Practice Partnerships (RPPs) have explored 
how to design context-specific and embedded forms of research that work with practitioners 
to evaluate programs and continuously improve implementation. These initiatives, while 
promising, pose a very different view of research—and its connection to practice—than the 
expert-driven model. As Bill Penuel and Caitlin Farrell contend: 


...While traditional research and development imagines a one-way path from re- 
search to practice, partnerships are a two-way endeavor, with practice informing 
the questions researchers ask and making research more relevant. In partner- 
ships, researchers and educators work together to search for and test solutions 
to practice, blending ideas and evidence from research with the wisdom of prac- 
tice.3” 


Such partnerships, they argue, offer ways to reimagine ESSA’s commitment to evidence-based 
practice. These partnerships also focus on the use of evidence, understood as an “an inter- 
active process that is most likely to take place when there are sustained opportunities for 
interactions between researchers and educators.”3° 


ESSA, while still wedded to very particular research designs, does transfer considerable 
authority for the design and implementation of evidence-based practice to states and 
districts. As implementation unfolds over the coming year, these groups will indeed need 
guidance for how to navigate the complex—and at times conflicting—views of the rela- 
tionship between research evidence and practice. Evidence-based policy is an easy thing 
to agree about: who is against evidence? But the harder conversations—and the questions 
to come—involve what counts as evidence, and who gets to determine the evidence that 
is most relevant to local contexts. While this report raises a number of important issues, 
it leaves this difficult and important work undone. But, new models and research do of- 
fer considerable insights for practitioners, in ways that may better address the challenges 
identified in this report. 
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